Toward Automated Component-Level Evaluation
نویسندگان
چکیده
Automated component-level evaluation of information retrieval is discussed. The advantages of such an approach are considered, as well as the requirements for implementing it. Acceptance of such systems by researchers is discussed.
منابع مشابه
Automated Component-Level Evaluation: Present and Future
Automated component–level evaluation of information retrieval (IR) is the main focus of this paper. We present a review of the current state of web–based and component–level evaluation. Based on these systems, propositions are made for a comprehensive framework for web service–based component–level IR system evaluation. The advantages of such an approach are considered, as well as the requireme...
متن کاملA Semi-automatic Evaluation Scheme
The evaluations of many NLP applications share a two-step procedure: the processing and aggregation of reference data to yield a gold-standard data set, and the subsequent comparison with peer system results. Traditionally, the former has been performed by human annotators and the latter has been conducted either manually or automatically. Moving toward a fully automated evaluation procedure, w...
متن کاملToward Revision-Sensitive Feedback in Automated Writing Evaluation
Revising is an essential writing process yet automated writing evaluation systems tend to give feedback on discrete essay drafts rather than changes across drafts. We explore the feasibility of automated revision detection and its potential to guide feedback. Relationships between revising behaviors and linguistic features of students’ essays are discussed.
متن کاملMetacognitive Beliefs and Students’ Tendency toward Drug Abbuse and Cross-level Effect of School-Bounding
Objective: The present research aimed to examine positive and negative beliefs about worry and tendency of students to drug abuse in terms of cross-level effect of school-bounding. Methods: In this multi-level investigation, 1000 students of high schools were selected by means of multi-stage sampling technique. Then, they completed metacognitive questionnaire (MCQ), school-bound...
متن کاملToward Automatic Role Identification in Unannotated Folk Tales
This paper presents an approach for automatically identifying high-level narrative structure information, particularly character roles, from unannotated folk tales. We introduce a new representation called action matrices to encode Propp’s narrative theory on character role and their “sphere of action.” We tested our approach in a fully automated system (Voz) using a corpus of 10 folk tales. Ou...
متن کامل